Skip to content

Phase 3: runtime-config seam cleanup to Phase 4 boundary#11

Merged
ethanj merged 60 commits intoarchitecture2from
feat/phase-3-config-audit
Apr 17, 2026
Merged

Phase 3: runtime-config seam cleanup to Phase 4 boundary#11
ethanj merged 60 commits intoarchitecture2from
feat/phase-3-config-audit

Conversation

@ethanj
Copy link
Copy Markdown
Contributor

@ethanj ethanj commented Apr 16, 2026

Summary

  • finish the remaining phase-3 config-threading cleanup needed for an honest Phase 4 boundary
  • thread ingest, service, route, and lineage behavior through explicit runtime-owned seams instead of hidden singleton reads
  • add focused regression tests and truthfulness guards so the narrowed seam stays honest

Included work

Service and ingest runtime-config threading

  • 75c670e refactor(service): allow explicit config injection
  • 8953420 test(service): prove explicit config reaches ingest path
  • bab4d6c refactor(runtime): pass config explicitly into MemoryService
  • 31fe6f4 test(service): prove explicit config reaches quick-ingest path
  • 246654b test(service): prove explicit config reaches workspace-ingest path
  • 19b985f thread ingest-time link generation config through runtime seam
  • 2e63c38 refactor(ingest): thread explicit ingest config seam
  • 89dfdbd refactor(ingest): thread entropy and composite config
  • aa3e646 thread namespace classification through runtime config seam

Route and search seam cleanup

  • cd6e8ce refactor(search): narrow runtime config seam types
  • afc15ce thread route-layer config reads through injected adapter

Lineage cleanup and regression gates

  • efe9634 refactor(lineage): remove config singleton from memory-lineage.ts
  • 56f37a6 test(lineage): prove consolidation forwards runtime config
  • 560caa6 test: add config singleton import regression gate
  • 58faf40 fix(test): broaden config singleton gate to catch multi-import patterns
  • 9a8c286 fix(test): use file-level multiline matching in config singleton gate
  • 903a409 docs(test): refresh config seam truthfulness notes

Supporting test realism

  • fe24be4 fix(test): add missing compositeMaxClusterSize to config mock
  • dbcd210 test: add max-cluster-size cap assertion for compositeGrouping

Validation

  • dotenv -e .env.test -- npx vitest run src/services/__tests__/memory-ingest-runtime-config.test.ts src/services/__tests__/memory-storage-runtime-config.test.ts src/services/__tests__/memory-lineage-runtime-config.test.ts src/services/__tests__/memory-crud-runtime-config.test.ts src/__tests__/memory-route-config-seam.test.ts src/__tests__/config-singleton-audit.test.ts src/app/__tests__/runtime-container.test.ts src/services/__tests__/composite-grouping.test.ts --reporter verbose
  • dotenv -e .env.test -- npx tsc --noEmit
  • earlier search/runtime seam validations still included in this branch remain relevant, especially:
    • dotenv -e .env.test -- npx vitest run src/services/__tests__/query-augmentation.test.ts src/services/__tests__/query-expansion.test.ts src/services/__tests__/memory-search-runtime-config.test.ts src/services/__tests__/search-pipeline-runtime-config.test.ts --reporter verbose

Reviewer orientation

  • This PR has grown well past the original request-time retrieval slice. It is now the live Phase 3 checkpoint for config-threading honesty before the first true Phase 4 work.
  • The important endpoint of this branch is not “all config reads are gone.” It is that the Phase-4-adjacent ingest, service, route-behavior, and lineage paths now thread runtime-owned config explicitly where behavior/provenance would otherwise lie.
  • Remaining singleton-backed reads in src/routes/memories.ts are concentrated in the default runtime snapshot adapter used for diagnostic reporting (/memories/config, /memories/health). Request-path behavior is already routed through the injected adapter.

Non-claims

  • does not implement Phase 4 hive-links/schema work
  • does not add schema or migration changes
  • does not claim full repo-wide singleton eradication
  • does not claim full-suite green; validations above are the focused proof bundle for this seam cleanup
  • does not claim production route diagnostics are fully decoupled from the config source of truth

ethanj and others added 29 commits April 16, 2026 07:13
Introduce an explicit, isolated composition root for Atomicmemory-core
and remove the last module-level mutable wiring (setEntityRepository).
Part of the rearchitecture plan at
atomicmemory-research/docs/atomicmemory-core-rearchitecture-plan-2026-04-16.md.

New composition layer (src/app/):
- runtime-container.ts: createCoreRuntime({ pool, config? }) composes
  repositories and services from explicit deps. Pool is required — the
  composition root is a pure function with no side-effectful singleton
  imports at module load. Config defaults to the module-level value
  because it is pure env-derived data.
- startup-checks.ts: checkEmbeddingDimensions() returns a structured
  result instead of calling process.exit.
- create-app.ts: createApp(runtime) builds the Express app, wiring
  routers and middleware onto a runtime container.
- __tests__/runtime-container.test.ts: 9 composition tests covering
  container wiring, startup-check branches, and the app factory.

server.ts: thinned from 101 → 71 lines. Imports the singleton pool once
and passes it to createCoreRuntime({ pool }). Shutdown closes
runtime.pool so a custom bootstrap closes the right pool. Preserves all
original named exports (app, service, repo, claimRepo, trustRepo,
linkRepo) plus the new `runtime` export.

search-pipeline.ts: removed module-level entityRepo and the
setEntityRepository() setter. Threaded entityRepo through as an
explicit parameter to runSearchPipelineWithTrace and all ten internal
helpers that consume it. The one call site in memory-search.ts passes
deps.entities.

Test fixtures and current-state-retrieval-regression dropped their
references to the removed setEntityRepository export.

Preserved:
- Endpoint behavior unchanged, routes unchanged, MemoryService kept as
  a thin compatibility wrapper.
- No DI framework, no package split, no config/index/route churn.

Validation:
- npx tsc --noEmit: clean
- Targeted run (composition, current-state retrieval regression,
  route-validation, smoke): 19/19 pass
- Full test suite: 884 pass / 1 pre-existing fail (deployment-config
  regex predates this change; confirmed via stash against main)
- fallow --no-cache: 0 above threshold, maintainability 90.9

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…me (phase 1a.5)

Codex stop-time review flagged the runtime container's `config?`
override as dishonest: the override only influenced repo construction
(entityGraphEnabled, lessonsEnabled), while routes/, services/, and the
search pipeline still read config directly from the module singleton at
25+ call sites. A consumer passing a custom config would silently get
split-brain behavior — repos honoring the override, everything else
ignoring it.

Scope decision:
- Preferred path (thread runtime.config through createMemoryRouter and
  MemoryService deps) was scoped and rejected for Phase 1A.5: any
  honest thread requires reaching memory-search.ts (4 reads) which
  delegates to search-pipeline.ts (35 reads), which is Phase 1B work.
  Stopping short would preserve the split-brain condition the reviewer
  flagged.
- Safest honest fallback: remove the override. Keep runtime.config as
  a stable reference to the module singleton so consumers still have a
  single named entry point for config, but drop the promise that
  `createCoreRuntime` can take a different config.

Changes:
- CoreRuntimeDeps: drop `config?: CoreRuntimeConfig`. Only `pool` is
  accepted. JSDoc documents why: most config reads happen below this
  layer, so an override here would be silently ignored.
- createCoreRuntime: read `config` from the module singleton directly
  for repo-construction flags. Single source of truth.
- CoreRuntimeConfig type kept as `typeof config` with a comment
  flagging Phase 1B as the place proper config splitting + per-runtime
  override will land.
- Tests: the flag-branch coverage (`entityGraphEnabled`,
  `lessonsEnabled`) now uses a small `withConfigFlag` helper that
  temporarily mutates the module singleton and restores on cleanup.
  That is an honest reflection of how the app actually works — config
  mutation via the live singleton is already the mechanism behind
  `PUT /memories/config`. Added one explicit test that
  `runtime.config` references the module singleton, so future
  drift is caught.

Validation:
- npx tsc --noEmit: clean
- src/app/__tests__/runtime-container.test.ts: 9/9 pass
- Full suite: 884 pass / 1 pre-existing fail (deployment-config regex)
- fallow --no-cache: 0 above threshold, maintainability 90.9

Follow-ups deferred to Phase 1B:
- Thread config through createMemoryRouter, MemoryService deps, and
  the search pipeline so per-runtime config overrides become honest.
- Split config into CoreRuntimeConfig (public) + InternalPolicyConfig
  (experimental), per the rearchitecture plan.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PR #6 changed docker-compose.yml ports from "3050:3050" to
"${APP_PORT:-3050}:3050" for side-by-side CI safety. The regex in
deployment-config.test.ts:105 was written for the bare-digit shape and
silently failed on the env-var substitution form, leaving the app-port
exposure assertion broken since that PR landed.

Update the assertion to use a new composePortBindingRegex(internalPort)
helper that accepts either a literal external port or a
${VAR:-default} shell-variable substitution. The helper makes the
pattern intent explicit and prevents the same drift from recurring on
future compose changes.

Restores the deployment-config test suite to a clean baseline (15/15
pass). Recommended in the Phase 1A follow-up analysis as the smallest
move to restore baseline integrity before Phase 1B.

Known follow-up: src/__tests__/route-validation.test.ts
"skip_extraction (storeVerbatim)" returns 500 in some local
environments. Out of scope for this fix; investigate separately before
declaring full suite green.

Validation:
- npx tsc --noEmit: clean
- npx vitest run src/__tests__/deployment-config.test.ts: 15/15 pass
- fallow --no-cache: 0 above threshold, maintainability 90.9 unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Memo-only artifact in docs/design/. Documents the source-grounded
audit that informed the Phase 1A refactor (ff57540, 0da468e) and the
minimum safe sequence for Phase 1B onward.

Contents:
- Hazards that existed before Phase 1A: search-pipeline module-level
  entityRepo with setter, server.ts as singleton composition root
  with bootstrap-on-import, parallel DI paths for the entity repo
- What Phase 1A resolved: runtime-container composition root,
  createApp factory, startup-checks extraction, module-global
  removal, search-pipeline functions now take entityRepo as a param
- What Phase 1A deliberately did NOT do: config override on
  createCoreRuntime (53 call sites still import config directly),
  process-global handler extraction, server.ts re-export surface
- Remaining hazards post-Phase 1A, with exact file:surface pointers
- Files and tests to watch during Phase 1B+ churn
- Phase 1B through 1E follow-on sequence with independently testable
  and reversible exit criteria

Source-grounded throughout — every reference is either a commit SHA
or a path within src/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…composed boot

Memo-only artifact in docs/design/. Records the one integration test
still missing after 0da468e: boot createApp(createCoreRuntime(...))
and prove config-facing HTTP behavior matches the singleton-backed
runtime end-to-end.

Contents:
- What is missing, in one sentence
- Why runtime-container.test.ts is insufficient — source-grounded at
  src/app/__tests__/runtime-container.test.ts lines 17, 42-80, 119-126
- Why route-validation.test.ts is insufficient — it bypasses
  createCoreRuntime and createApp by design
- Test shape (not implementation): boots app.listen(0), at least one
  write + read HTTP round-trip through the composed graph, parity
  assertion against a reference
- Narrow acceptance criteria (6 items, each binary)
- Explicit non-goals (route-validation, per-endpoint business logic,
  config-threading parity across 53 sites — those are other tests)

No code, no test file. This memo only defines what "done" looks like
so the test can be written in a focused follow-up.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Closes the gap identified in docs/design/phase-1a-composed-boot-parity-test.md
after 0da468e: no live-HTTP test existed proving that
createApp(createCoreRuntime({ pool })) produces a server whose
observable behavior matches the singleton-backed reference.

The existing runtime-container.test.ts is composition-shape only
(stubbed pool, no SQL, no requests issued). The existing
route-validation.test.ts is route-logic only (deliberately bypasses
the composition seam, hand-wires its own MemoryService). Neither
covered the seam itself.

This test boots two ephemeral Express listeners — one composed via
the Phase 1A seam, one hand-wired in the legacy shape — and asserts:

1. GET /memories/health returns identical config payloads from both,
   proving config flows correctly through composition into routes.
2. GET /memories/stats returns identical stats from both, proving the
   full route → service → repo → pool graph is connected end-to-end
   through the composition seam (this query actually hits the DB).

Two requests are sufficient: more would shade into per-endpoint
behavior coverage, which is route-validation.test.ts territory and
explicitly out of scope per the design doc's acceptance criteria.

Validation:
- npx tsc --noEmit: clean
- vitest run src/app/__tests__/composed-boot-parity.test.ts: 2/2 pass
- vitest run src/app/__tests__/: 11/11 pass (no regression in
  runtime-container.test.ts)
- fallow --no-cache: 0 above threshold, maintainability 90.9 unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
codex2 review of ba7fd6e flagged the remaining gap: composed-boot
parity covered config-facing READS (GET /memories/health,
GET /memories/stats) but not the config-facing WRITE seam
(PUT /memories/config), which is the one route that mutates the
module-level config singleton end-to-end.

Add a single test case to the existing composed-boot-parity file:

1. PUT /memories/config on the composed app with a sentinel
   max_search_results value (current + 17, safely past validation
   floor).
2. Assert the PUT response surfaces { applied, config } with the
   sentinel applied.
3. GET /memories/health on BOTH the composed app and the reference
   app and assert both surface the sentinel — proving the composed
   write seam mutates the same singleton the reference app reads.
4. finally{} restores the original value via direct config mutation
   (not a follow-up PUT) so cleanup does not depend on either server
   still being healthy at teardown.

Test ordering is deliberate: this test runs LAST in the file so any
finally{} hiccup cannot bleed sentinel state into the existing GET
parity tests above.

Validation:
- npx tsc --noEmit: clean
- vitest run src/app/__tests__/composed-boot-parity.test.ts: 3/3 pass
- vitest run src/app/__tests__/{composed-boot-parity,runtime-container}.test.ts: 12/12 pass (no regression)
- fallow --no-cache: 0 above threshold, maintainability 90.9 unchanged
- git diff --stat: 1 file changed, +39/-0

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…h paths

Completes the retrieval-policy signature migration by wiring
CoreRuntimeConfig from the MemoryService deps into the 5 call sites
that were still using the static module-level config import.

- search-pipeline.ts:
  - Add runtimeConfig?: CoreRuntimeConfig to SearchPipelineOptions
    (falls back to static config when omitted for non-migrated callers)
  - runSearchPipelineWithTrace derives policyConfig once and passes to
    resolveRerankDepth, applyRepairLoop, and applyAgenticRetrieval
  - applyRepairLoop now takes policyConfig as a required param and uses
    it for shouldRunRepairLoop and shouldAcceptRepair

- agentic-retrieval.ts:
  - applyAgenticRetrieval accepts optional policyConfig (defaults to
    static config) and threads it through retrieveSubQueries
  - Both mergeSearchResults call sites now receive policyConfig

- memory-search.ts:
  - executeSearchStep now passes deps.config as runtimeConfig when
    invoking runSearchPipelineWithTrace

Scope intentionally narrow: only the 5 retrieval-policy call sites.
Other static-config uses in search-pipeline (mmrEnabled,
iterativeRetrievalEnabled, agenticRetrievalEnabled, crossEncoderEnabled,
etc.) are out of scope for this commit — separate tracking issue.

Validation:
- npx tsc --noEmit: clean
- vitest src/services/__tests__/retrieval-policy.test.ts: 46/46 passing
- pnpm test (full suite): 890/890 passing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The base schema.sql is idempotent (`CREATE TABLE IF NOT EXISTS`), so
re-running it cannot change the vector(N) dim of an existing
memories.embedding column. When the test DB was previously
initialized with a different EMBEDDING_DIMENSIONS (e.g. left over
from an earlier run with a different .env.test), every test that
inserts an embedding fails at the DB with an opaque 500.

The POST /memories/ingest/quick skip_extraction test in
route-validation.test.ts was the concrete victim: it passed once
the DB dim matched config, and failed with `expected 500 to be 200`
when the dim drifted.

Fix is scoped to the test harness:

- src/db/__tests__/test-fixtures.ts: setupTestSchema now queries the
  existing memories.embedding column dim. If it is set and does not
  match config.embeddingDimensions, drop and recreate public schema
  before applying schema.sql. If unset or matching, behavior is
  unchanged — no cost on the happy path.
- src/__tests__/route-validation.test.ts: swap the inline
  readFileSync + pool.query(schema) dance for the shared
  setupTestSchema helper so the new drift guard actually runs; drop
  now-unused config/readFileSync/path imports and __dirname.

Validation (this run):
- Drifted the DB to vector(1024), config at 1536.
- Targeted run of the skip_extraction test: passes. Post-test DB
  column dim is 1536 (realigned).
- Full route-validation.test.ts: 9/9 passing.
- All other setupTestSchema consumers (entity-graph,
  canonical-memory-objects, claim-slot-backfill,
  consolidation-execution): 30/30 passing.
- tsc --noEmit: clean.
- fallow --no-cache: 0 above threshold, maintainability 90.9 (good).

No product code changed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…-retrieval

agentic-retrieval.ts still imported the static config singleton after
the phase-3 retrieval-policy threading work (8cc2aae, 8e763ef). Two
remaining references:

- retrieveSubQueries: read config.hybridSearchEnabled → read from the
  already-threaded policyConfig param
- applyAgenticRetrieval: policyConfig = config default fallback → make
  the param required

Both changes are safe because applyAgenticRetrieval has a single caller
(search-pipeline.ts:187) which already passes policyConfig. The sole
behavioral change is where hybridSearchEnabled is read from — now from
the runtime-owned config that flows down from MemoryService deps,
instead of the static module singleton. Same values either way under
current wiring.

Static config import removed entirely from this file. Zero module-global
config dependence remaining; only stale mention is a JSDoc line.

sourceSite?: / referenceTime?: optionals changed to | undefined because
TS requires non-trailing optionals to be nullable explicitly when a
later positional param becomes required. Same runtime shape.

Validation:
- npx tsc --noEmit: clean
- vitest src/services/__tests__/retrieval-policy.test.ts: 46/46
  (direct coverage of mergeSearchResults, the retrieval-policy
   function agentic-retrieval calls)
- pnpm test (full suite): 82 files, 890 tests passing, 14.65s

Explicit non-claims:
- Does NOT remove static config use from search-pipeline.ts (still uses
  it for mmrEnabled, iterativeRetrievalEnabled, agenticRetrievalEnabled,
  crossEncoderEnabled, rerankSkipTopSimilarity, rerankSkipMinGap,
  hybridSearchEnabled, entityGraphEnabled, etc.) — separate slice.
- Does NOT add a unit test for applyAgenticRetrieval itself — none
  exists today and adding one would require mocking llm + embeddings,
  outside narrow cleanup scope.
- Does NOT prove runtime equivalence via eval/benchmark — only
  type-level and unit-test equivalence. A live integration run against
  a real retrieval scenario would be the strongest check.
- Does NOT touch the JSDoc at line 130 that still says "agenticRetrievalEnabled
  is true in config"; that reference is documentation of behavior
  decided upstream in search-pipeline.ts, not this module's concern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codex stop-review flagged that commit 2fc5633 made applyAgenticRetrieval's
policyConfig param required, which is a breaking change for an exported
function. Restored the `= config` default and re-imported the static
config to support it.

Net effect vs pre-session baseline (8e763ef) is now one behavioral line:

  retrieveSubQueries: config.hybridSearchEnabled → policyConfig.hybridSearchEnabled

The static `config` import is retained solely as the default fallback
when no policyConfig is passed. The single in-repo caller
(search-pipeline.ts:187) always passes policyConfig, so the fallback is
only exercised by external/future callers that haven't migrated — which
preserves backward compatibility per the stop-review guidance.

Validation:
- npx tsc --noEmit: clean
- vitest src/services/__tests__/retrieval-policy.test.ts: 46/46
- pnpm test (full suite): 82 files, 890 tests passing, 14.01s

Revised non-claims:
- Static config import is NOT fully removed; it remains as the default
  fallback. The narrow reduction delivered is the one hybridSearchEnabled
  read now going through policyConfig when threaded.
- Other non-claims from 2fc5633 still apply (no new unit test,
  no eval-level runtime equivalence proof, JSDoc not retouched).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Swap the agentic-retrieval enablement check in runSearchPipelineWithTrace
from the module-level config singleton to the already-derived policyConfig
(which falls back to the singleton when options.runtimeConfig is absent).
Behavior is preserved; this just pairs the gate with the retrieval call
whose internals were already narrowed in 060501b.
Swap the iterative-retrieval enablement check in runSearchPipelineWithTrace
from the module-level config singleton to the already-derived policyConfig
(which falls back to the singleton when options.runtimeConfig is absent).
Symmetric with the agentic-retrieval gate migrated in 0c60d06 — behavior
is preserved.
Swap the mmrEnabled read that determines mmrPoolMultiplier (and therefore
candidateDepth) in runSearchPipelineWithTrace from the module-level config
singleton to the already-derived policyConfig, which falls back to the
singleton when options.runtimeConfig is absent. Behavior is preserved.

Note: the remaining mmrEnabled reads inside applyExpansionAndReranking are
intentionally untouched here — those are inner-function reads that require
a separate signature-threading slice.
Centralize the existing lineage-producing write paths behind a single
internal seam and route canonical add, claim backfill, AUDN
update/supersede/delete, and consolidation through it without changing
schema, mutation semantics, or workspace/scope behavior.
Swap the hybridSearchEnabled read in the 'initial' trace.stage metadata
from the module-level config singleton to the already-derived policyConfig,
which falls back to the singleton when options.runtimeConfig is absent.
This is trace-reporting only; retrieval behavior is unaffected and the
behavioral hybrid reads in runInitialRetrieval / maybeApplyAbstractHybridFallback
/ applyRepairLoop remain on the static singleton.
Add a narrow integration assertion that consolidation still emits claim
lineage without creating a mutation canonical memory object or attaching
`cmo_id` to the consolidated projection.
Add an optional policyConfig parameter (defaulting to the module-level
config singleton) to runInitialRetrieval, and pass the already-derived
policyConfig from runSearchPipelineWithTrace. The behavioral
hybridSearchEnabled read inside runInitialRetrieval now reads from
policyConfig instead of the module singleton.

Behavior is preserved: when callers don't pass policyConfig, the default
argument falls through to the same static config that was being read
directly before. The lone call site in runSearchPipelineWithTrace now
passes the runtime-owned policyConfig explicitly.

Unchanged by this slice:
- maybeApplyAbstractHybridFallback still reads config.hybridSearchEnabled
  and config.entityGraphEnabled directly
- applyRepairLoop still reads config.hybridSearchEnabled at its inner
  runMemoryRrfRetrieval call
- All query-expansion / entity-graph / link-expansion / reranker /
  cross-encoder / PPR / MMR-reranker reads remain on the static config
Add a focused AUDN delete integration test that asserts the preserved
delete-tombstone claim-version row fields directly, including null
memory_id, deleted-prefix content, zero importance, blank source
fields, prior-version linkage, and reused embedding.
…llback

Add an optional policyConfig parameter (defaulting to the module-level
config singleton) to maybeApplyAbstractHybridFallback, and pass the
already-derived policyConfig from runSearchPipelineWithTrace. The two
short-circuit reads inside the helper — hybridSearchEnabled and
entityGraphEnabled — now read from policyConfig instead of the module
singleton.

Behavior is preserved: when callers don't pass policyConfig, the default
argument falls through to the same static config that was being read
directly before. Symmetric with ce36f2b, which threaded policyConfig
into runInitialRetrieval.

Unchanged by this slice:
- applyRepairLoop still reads config.hybridSearchEnabled at its inner
  runMemoryRrfRetrieval call
- All query-expansion / entity-graph co-retrieval / link-expansion /
  reranker / cross-encoder / PPR / MMR-reranker reads remain on the
  static config
…rieval

Add an optional policyConfig parameter (defaulting to the module-level
config singleton) to applyEntityNameCoRetrieval, and pass the
already-derived policyConfig from runSearchPipelineWithTrace. The two
config reads inside the helper — entityGraphEnabled (gating) and
linkExpansionMax (budget) — now read from policyConfig instead of the
module singleton.

Behavior is preserved: when callers don't pass policyConfig, the default
argument falls through to the same static config that was being read
directly before. Symmetric with ce36f2b (runInitialRetrieval) and ed7af66
(maybeApplyAbstractHybridFallback).

Unchanged by this slice:
- applyRepairLoop inner config reads (hybridSearchEnabled, repair profile
  weights) remain on the static singleton
- applyQueryExpansion, applyQueryAugmentation, applyTemporalQueryExpansion,
  applyLiteralQueryExpansion, applySubjectQueryExpansion still read
  static config
- All reranker, cross-encoder, link-expansion, PPR, MMR-reranker, and
  generateLinks reads remain on the static singleton
Add a focused lineage test that asserts ensureClaimTarget backfill
creates a claim-version row with the preserved null provenance fields,
without changing the existing tombstone-first behavior or any
production logic.
Add an optional policyConfig parameter (defaulting to the module-level
config singleton) to applyQueryAugmentation, and pass the already-derived
policyConfig from runSearchPipelineWithTrace. The two gate reads inside
the helper — queryAugmentationEnabled and entityGraphEnabled — now read
from policyConfig instead of the module singleton.

Behavior is preserved: when callers don't pass policyConfig, the default
argument falls through to the same static config that was being read
directly before. Symmetric with ce36f2b (runInitialRetrieval), ed7af66
(maybeApplyAbstractHybridFallback), and b9df846 (applyEntityNameCoRetrieval).

Unchanged by this slice:
- applyRepairLoop inner config reads (hybridSearchEnabled, repair profile
  weights) remain on the static singleton — deliberately out of scope
- applyQueryExpansion, applyTemporalQueryExpansion, applyLiteralQueryExpansion,
  applySubjectQueryExpansion still read static config where applicable
- applyExpansionAndReranking, runMemoryRrfRetrieval, expandViaPPR,
  expandViaEntities, expandWithLinks, generateLinks still read static config
Keep the lineage seam tests focused by collapsing the repeated legacy
backfill setup into one helper. This is test-only and preserves the
existing tombstone/backfill assertions.
@ethanj ethanj changed the title Phase 3A: audit config-singleton import surface Phase 3A-B: runtime threading and lineage seam hardening Apr 17, 2026
ethanj added 22 commits April 16, 2026 22:41
Add the symmetric retrieval-format fence proving an explicit
stagedLoadingEnabled=false option overrides an enabled module config.
Keep the slice test-only and scoped to packaging truthfulness.
Let MemoryService accept an optional runtime config for its delegated
modules while preserving the singleton default. Add a focused unit test
covering both the explicit override and current default behavior.
Add one test in memory-service-config.test.ts asserting that a
MemoryService constructed with an explicit runtimeConfig threads
that config into performIngest via deps.config.

This closes the search→ingest confidence gap: the existing tests
proved the override reaches performSearch (75c670e), and
19b985f proved performIngest threads deps.config into generateLinks.
This new test proves the middle link — MemoryService passes the
explicit config into the ingest deps, not just the search deps.

Reuses the existing mock scaffolding: promotes performIngest to a
hoisted mock (mockPerformIngest) so the test can assert against it,
matching the existing mockPerformSearch pattern. No source changes.
Replace the broad CoreRuntimeConfig alias with an explicit interface
covering the runtime-container, startup-check, search/runtime, and
MemoryService config seams already threaded today. Keep the contract
honest without widening route behavior or claiming full runtime-wide
config injection.
Replace broad CoreRuntimeConfig usage in the already-threaded search
runtime seam with narrower local config contracts for search-pipeline and
agentic-retrieval. Preserve behavior and keep the slice type-only.
Keep the composition root honest by having createCoreRuntime pass the
module-level config singleton explicitly into MemoryService instead of
relying on the service constructor's fallback default.
Static test that counts non-test source files importing the module-level
config singleton and fails if the count exceeds 32 (current baseline).
As config-threading PRs land and remove singleton imports, the threshold
should be ratcheted down — that one-line constant change is the explicit
friction that prevents regression.

Includes a staleness check that warns (but does not fail) when the
threshold has more than 5 files of slack, prompting the next threading
PR to tighten the gate.

Non-overlapping with the in-flight ingest/storage/audn/lineage config
threading on this branch. Uses the static file-reading pattern from
deployment-config.test.ts — no runtime, no DB, no mock dependencies.
The initial grep (`import { config } from`) missed three real singleton
imports that use multi-binding forms:
  - src/routes/memories.ts: `import { config, updateRuntimeConfig, ... }`
  - src/services/reranker.ts: `import { config, type CrossEncoderDtype }`
  - src/index.ts: `export { config, ... } from './config.js'`
  - src/app/runtime-container.ts: `import { config } from` (composition root)

Replaces the simple grep -rl with a regex that matches any import or
export statement binding the `config` value from a config.js path,
excluding `import type`-only lines. Threshold updated from 32 to 36
to reflect the accurate count.
The prior line-by-line grep+regex approach would miss multiline import
blocks like:
  import {
    config,
    updateRuntimeConfig,
  } from '../config.js'

Replaces the grep pipeline with readFileSync + a regex using the /s
(dotAll) flag that matches across newlines. The test now:
- Walks src/ recursively (skipping __tests__ and node_modules)
- Reads each .ts file as a string
- Matches import/export blocks binding `config` from a config path
- Filters out `import type`-only statements

Verified against 6 representative patterns: single-line, multi-binding
single-line, multiline import, multiline re-export, type-only (skip),
and non-config-binding (skip). Count remains 36; no false positives or
false negatives vs the perl reference.
The composite-grouping test mock omitted compositeMaxClusterSize,
causing it to resolve to undefined. Since `n >= undefined` is false
in JS, the cluster-size cap in clusterBySimilarity was silently
disabled — tests never exercised the capping behavior.

Adds compositeMaxClusterSize: 3 (matching the production default)
and adjusts the L1 overview test to use multi-sentence facts within
the 3-fact cap so the joined content still exceeds the truncation
threshold.
The last caller not passing config — consolidation-service.ts:139 —
now passes { claims, config } into emitLineageEvent. With all six
callers supplying config explicitly, LineageDeps.config is made
required (was optional), the lineageActorModel fallback to the module
singleton is removed, and the `import { config }` is dropped.

memory-lineage.ts is now fully config-singleton-free. Regression gate
threshold ratcheted from 36 to 35.
fe24be4 restored compositeMaxClusterSize to the test mock but had no
test exercising the cap. Adds one case: 5 similar facts with cap=3,
asserts no cluster exceeds 3 members and all 5 facts are accounted
for across the resulting composites.
@ethanj ethanj changed the title Phase 3A-B: runtime threading and lineage seam hardening Phase 3: runtime-config seam cleanup to Phase 4 boundary Apr 17, 2026
Base automatically changed from feat/phase-2b-observability-contract to architecture2 April 17, 2026 09:00
@ethanj ethanj merged commit 874e4b2 into architecture2 Apr 17, 2026
@ethanj ethanj deleted the feat/phase-3-config-audit branch April 17, 2026 10:03
ethanj added a commit that referenced this pull request Apr 19, 2026
Cuts over `main` to the Phase 1A–7 rearchitecture (composition root,
  explicit scope/observability contracts, store-narrowed repository access,
  public consumption seams, config split, leaf-module config threading,
  retrieval orchestration polish) plus the OSS-release-prep on top.

  ⚠️ Breaking changes for HTTP / SDK consumers
  - All API endpoints are now mounted under `/v1` (e.g.
    `POST /v1/memories/ingest`, `PUT /v1/agents/trust`). The unversioned
    `/health` liveness probe is unchanged.
  - Workspace `GET /memories/list`, `GET /memories/:id`, and
    `DELETE /memories/:id` now require `agent_id` when `workspace_id`
    is present; missing returns 400 (no visibility-unsafe fallback).
  - `PUT /memories/config` returns 410 Gone in production. Provider/model
    fields (embedding_provider, embedding_model, llm_provider, llm_model)
    are rejected with 400 — they were never honored mid-flight (provider
    caches are fixed at first use). Set via env at process start.
  - npm package renamed `@atomicmemory/atomicmemory-engine` →
    `@atomicmemory/atomicmemory-core`. Tarball now ships `dist/` (built
    via `tsc`); `main`/`types`/`exports` point at compiled output.
  - Deep-importers of `services/embedding` and `services/llm` must call
    `initEmbedding(config)` / `initLlm(config)` before hot-path APIs.
    Consumers using `createCoreRuntime({ pool })` are auto-initialized.

  Rearchitecture (Phases 1A–7)
  - Phase 1A: composition root via `createCoreRuntime` + `createApp` (#8)
  - Phase 2A: canonical search scope contract (#9)
  - Phase 2B: explicit retrieval observability contract (#10)
  - Phase 3: runtime-config seam cleanup to Phase 4 boundary (#11)
  - Phase 4: ingest pipeline decomposition (475 → 215 lines) (#13)
  - Post-Phase 4: unify scope contract via `scopedSearch`/`scopedExpand`,
    document schema scoping gaps as deferred (#15)
  - Phase 5: narrow repository access behind 8 domain-facing stores;
    workspace ingest now flows through canonical lineage (#16)
  - Phase 6: publish stable consumption seams (HTTP, in-process, docker)
    with two-direction parity contract test (#17)
  - Phase 7 Steps 3a–3c: split runtime config into supported/internal
    partitions; deprecate `PUT /memories/config` for production (#18)
  - Phase 7 Step 3d: thread config through 5 leaf modules (33→28
    singleton audit) (#21)
  - Phase 7 Item 4: retrieval polish — `memory-search.ts` reduced to
    pure orchestration (374 → 248 lines, -34%) (#22)
  - Chore: reduce fallow duplication 367 → 234 lines (#12)

  OSS release prep
  - `"private": true` removed; package renamed to
    `@atomicmemory/atomicmemory-core`. `files` field scopes the tarball.
  - `tsconfig.build.json` + `prepublishOnly` so `npm publish` always ships
    compiled `dist/`. Bare-import smoke test passes.
  - `release.yml` publishes to public npm on tag push (NPM_TOKEN secret).
  - SuperMem codename scrubbed from `src/`, tests, docker-compose, and
    `.env.example` (DB user/name/password renamed to `atomicmemory`).
  - Private-research-repo URLs unlinked from public docs.
  - README links to docs.atomicmemory.ai.
  - `/v1` API prefix on all routes; mount-coverage test added.
  - CI workflow: set `CORE_RUNTIME_CONFIG_MUTATION_ENABLED=true` to
    match `.env.test` (gitignored) and unblock the composed-boot-parity
    test on `PUT /v1/memories/config`.

  Verification
  - 966/966 tests pass (100 files)
  - npx tsc --noEmit clean
  - fallow --no-cache: 0 above threshold (maintainability 91.0)
  - npm publish --dry-run succeeds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant